• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    A method for producing controlled disturbances of the activity of an automated processing device during a test scenario of an application implemented by the automated processing device, characterized in that it comprises the following steps: - Acquisition of a configuration defining at least one perturbation by: - an identifier of the device to be disturbed, - a start date of the disturbance, - a duration for the disturbance, - a type for the perturbation, - a disturbance level, - setting place at least one disturbance generator according to the configuration of the at least one disturbance Launch of the test scenario
    公开号:FR3023940A1
    申请号:FR1456868
    申请日:2014-07-17
    公开日:2016-01-22
    发明作者:Christophe Germain；Natacha Bagnrd；Bernard Roy
    申请人:Bull SA；
    IPC主号:
    专利说明:

    [0001] The present invention relates to a method for producing controlled disturbances of the activity of an automated processing device during a test scenario of an application. TECHNICAL FIELD OF THE INVENTION [0001] The subject of the present invention is a method for producing controlled disturbances of the invention. activity of an automated processing device during a test scenario of an application. The field of the invention is the field of computer infrastructure performance testing, and more particularly the generation of additional activity that can be applied during a firing load. [0003] For the purposes of this document, load generation will be understood to mean the implementation of a test scenario according to the application's business procedures. A tool conventionally used for such a purpose is, for example, JMeter.
    [0002] STATE OF THE PRIOR ART [0004] During testing and profiling campaigns, an injection kit is used to perform load shots to measure the performance of an application chain. These tests are generally performed in a recipe / integration environment. That is to say in a controlled environment and protected from outside influences. This method makes it possible to ensure that the measured performances are only those of the information system subjected to the performance tests. However, in its real context, the information system is not isolated. The results of the performance tests are then not relevant to decide what should be the configuration to use in an open production environment. A current solution is to perform a load test corresponding to an estimated peak solicitation of the application. This test will give a measure of performance. From this measurement and from a safety coefficient, it is possible to calculate, by extrapolation, a configuration that we hope to be resistant but without being able to guarantee it.
    [0003] SUMMARY OF THE INVENTION [0007] The object of the invention is to remedy all or some of the disadvantages of the state of the art identified above, and in particular to propose means for making it possible to guarantee the resistance of a configuration. The present invention can automatically simulate an activity on the server components of the tested chain. This simulation of activity is generated according to 4 axes making it possible to stress various elements of the system: - CPU - Read / write disk - Read / write network - Memory [0009] For each server of a chain applicative, it is possible to parameterize through a configuration of stress axes with for each of them a rate applied, the start time and duration. It is thus capable of producing an activity other than that resulting from the application itself and to obtain test results much closer to a real operation. For this purpose, one aspect of the invention relates to a method for producing controlled disturbances of the activity of an automated processing device during a test scenario of an application implemented by the device of the invention. automated processing, characterized in that it comprises the following steps: - Acquisition of a configuration defining at least one disturbance by: - An identifier of the device to be disturbed, - A start date of the disturbance, - A duration for the disturbance , - A type for the disturbance, - A disturbance level, - Establishment of at least one disturbance generator according to the configuration of the at least one disturbance - Launch of the test scenario. In addition to the main features which have just been mentioned in the preceding paragraph, the method / device according to the invention may have one or more additional characteristics among the following, considered individually or according to the technically possible combinations: A perturbation is among the list of at least: - CPU load, - Read / Write disk, - Read / Write network, - Memory utilization; according to the type of the disturbance, it comprises a calibration step of the disturbance generator; a disturbance start date is relative to a start date of the test scenario; - the generator is a process that knows its start date, has access to the current date and is controlled by a loop structure depending on the comparison of the difference between the current date and the start date with the duration of the disturbance; the loop includes instructions for: Disrupting the device according to the type of disturbance; Temporarily inhibiting the instructions to disturb in order to respect the required level of disturbance; - The implementation of the generator on a device comprises at least the following steps: - Copy of at least one executable file on the device, - Planning, on the device, the execution of at least one executable file copied [0013 The invention also relates to a digital storage device comprising a file corresponding to instruction codes implementing a method according to a combination of the characteristics of the preceding ones. The invention also relates to a device implementing a method according to a combination of the characteristics of the previous ones.
    [0004] BRIEF DESCRIPTION OF THE FIGURES [0015] Other features and advantages of the invention will emerge on reading the description which follows, with reference to the appended figures, which illustrate: FIG. 1, an illustration of means allowing the implementation of the method which is the subject of the invention; FIG. 2, an illustration of steps of the method according to the invention. For clarity, identical or similar elements are identified by identical reference signs throughout the figures. The invention will be better understood by reading the following description and examining the figures that accompany it. These are presented as an indication and in no way limitative of the invention. DETAILED DESCRIPTION OF AN EMBODIMENT [0018] FIG. 1 shows a test server device or test server 100 comprising: a microprocessor 110, storage means 120, typically a local or remote hard disk - A communication interface 130, for example an Ethernet network interface, FiberChannel, InfiniBand ... The microprocessor of the server device, the storage means of the server device and the communication interface of the server device are interconnected by a bus 150. When an action is taken to a device it is actually performed by a microprocessor of the device controlled by instruction codes stored in a memory of the device. If an action is taken to an application, it is actually performed by a microprocessor of the device in a memory of which the instruction codes corresponding to the application are recorded. When a device transmits or receives a message, this message is sent or received by a communication interface of the device. FIG. 1 shows that the test server 100 is connected to a network 200. [0022] FIG. 1 shows an application server 300 comprising: a microprocessor 310, storage means 320, an interface 330 of FIG. communication, - an internal clock 340. The microprocessor of the application server, the storage means of the application server, the communication interface of the application server, the internal clock of the application server are interconnected by a bus 350. The test server and the server The application means are connected via the network 200. [0025] FIG. 1 shows that the storage means of the test server comprise a zone 120.1 of disturbance configuration. Disturbance configuration area 120.1 is also called a disturbance configuration file or simply a disturbance configuration. A configuration file is a file, or a set of files, that allows you to associate multiple data or fields. This field association is also called a disturbance record. Such a disturbance configuration file is, for example, an XML file, a CSV file, an INI file, a JSON file .... A configuration file comprises zero, one or more configuration records. In a variant, the disturbance configuration file is stored in a database. A disturbance record includes at least the following fields: an identifier field of the device to be disturbed: for example a network address, or a resolvable name in network address; - A date of disturbance start field, it is: - an absolute date evaluated relative to an internal clock of a device producing the disturbance; - a date relative to the beginning of a test scenario; a disturbance duration field: for example a number of seconds; a typical perturbation field, for example a predetermined code making it possible to designate a perturbation among: processor load; of this field - read / write disc; - network read / write; - memory usage; a disturbance level field, the interpretation depending on the type of disturbance, it is for example: a percentage of maximum load for the processor or for the memory; - a bit rate for write readings; - an absolute value for the memory. The start dates, duration and level of disturbance are also called disturbance parameters. FIG. 1 also shows that the storage means of the test server comprise a zone 120.2 disturbance code comprising instruction codes corresponding to instruction codes for the implementation of disturbances. These instruction codes are either object codes, for example for disturbances related to memory, or so-called "shell" scripts or shell scripts. In one variant, the zone 120.2 disturbance code includes compilable instruction codes, the compilation being performed at the time of the establishment of disturbances. Disturbance code area 120.2 is itself subdivided, each of the subfields corresponding to at least one type of disturbance. In one variant, the disturbance codes can be parameterized to take into account perturbation parameters. Figure 2 shows a step 500 of acquiring a configuration. In step 500, the test server reads the contents of the disturbance configuration zone 120.1. In the implementation described zone 120.1 perturbation configuration corresponds to a file. In variants, the disturbance configuration zone 120.1 can be acquired by one or more requests sent to a database management system. Reading the zone 120.1 disturbance configuration allows the test server to obtain a list of disturbances to place, as well as the characteristics of these disturbances. These characteristics are at least those described in the description of the zone 120.1 perturbation configuration. Once the configuration acquired by the test server it goes to a step 510 of setting up disturbances on the target servers. Step 510 is implemented for each disturbance described in the configuration acquired in step 500 of acquisition. FIG. 2 shows that the step 510 of setting up a disturbance comprises: a step 510.1 of configuration of the disturbance: in this step the test server uses the type of the perturbation to extract from the zone 120.2 disturbance code the instruction codes corresponding to the type of disturbance. It is possible to set at this stage the duration of the disturbance, for example, by modifying the instruction codes of the disturbance according to the duration of the disturbance; a step 510.2 of deployment of the disturbance: in this step the test server copies on the application server identified by the device identifier field to disturb instruction codes of the perturbation; a step 510.3 for planning the disturbance: this step is done by issuing a scheduling command (typically an "at" command), the parameters of which are the disturbance start date and a command line. The command line includes an executable file and possibly parameters. The executable file is the one that has been deployed. In one variant, the parameters make it possible, for example, to set the duration of the disturbance. The deployment and planning steps are executed, for example, from the test server via a remote shell such as telnet or ssh. It is clear to those skilled in the art that the disturbances are deployed by a test server on application servers and then executed by the application servers according to a schedule managed by the application servers. If the start of disturbance dates are relative to the beginning of the test scenario, then, at the time of launching the scenario, the test server issues commands to reset the internal clocks of the devices to be disturbed. In practice, all the clocks of the servers are synchronized via the same NTP server. It is therefore easy, knowing the date of the beginning of the scenario, to fix dates for the disturbances. In this configuration the dates are either absolute or relative; in this case, the relative amount is added to the test start date during the planning stage. This synchronization facilitates the delegation of breadmaking management to the application servers by the test server. The step of setting up the disturbances is followed by a step 520 of launching the test scenario. Such a launch is, for example, an execution of the JMeter application. As far as the disturbances have been planned during the planning stage, these disturbances will run in parallel with the test scenario, according to the planning. A disturbance is similar to a loop whose pseudo code is the following: sdt = now (); as now () - sdt <disturb duration () [0040] It is seen with this code that the disturbance lasts only "duration", or duration is the duration of the disturbance. "Disturb ()" is a function that depends on the type of the disturbance. For example, for a perturbation of processor activity is for example a shell script corresponding to the following pseudo code: sdt = now () as now () - sdt <calculation duration () to sleep (x) [0042] Or: - "calculation ()" is a function whose purpose is to occupy the processor, for example the calculation of a tangent; - "sleep (x)" is a function that does not solicit the microprocessor for a period x, x being a parameter of the disturbance determined according to the desired level of disturbance. The duration is calculated before the beginning of the shot and guarantees the disruption of the CPU with the desired occupancy rate. In the case of the microprocessor disruption it may be useful to carry out a calibration, in particular to determine the waiting time. . To determine that it must be the value of the parameter to reach the desired level of disturbance. This is done by performing, for a known parameter value, a measure of the disturbance outside the test scenario. It is then possible, via an interpolation, to determine the value of the parameter to be used to reach the desired disturbance level. In a variant calibration is done by iteration, that is to say by varying the parameter until the activity measurement corresponds to the desired level of disturbance. Calibration is performed after the deployment, but before the launch of the test scenario. For a disruption read / write the pseudo code would be the following: sdt = now () total = 0 as now () - sdt <duration if total / (now () - sdt) <level write (file) total = total + size (file) [0046] An effective write is thus compared to a target level and one makes sure not to exceed durably this target level which is the desired level of disturbance. "Write" depends on the type of the disturbance: - disk: a "copy" - network instruction: a "netcat" instruction. [0047] "if total / (now () - sdt) <level" is an inhibition command. In a variant, this instruction includes a "sleep (x)" clause which makes it possible to reduce the processor footprint. These instructions, or commands, "copy" and "netcat" are such that they have a small footprint processor, that is to say that their implementation can not be considered a microprocessor disruption. A simple memory perturbation corresponds for example to the following pseudo code: sdt = now () m = allocation (level) as now () - sdt <sleep duration (x) release (m) [0050] In this case: - alloction (): is a memory allocation instruction, for example "malloc", - "free ()" is a free memory instruction, for example "free". This variant allows to occupy the memory statically, that is to say without moving the memory. In another variant, the following code is used which also makes it possible to simulate the use of the memory such that a program would use it. This vaiante requires an additional parameter which is either a default value or provided by the corresponding perturbation record. This record will then have an additional field, this field being a bitrate. The pseudo code is then: sdt = now () total = 0 source = array [] as now () - sdt <duration m = allocation (level) as total <level if total / (now () - sdt) < flow write (source, m) otherwise sleep () total = total + size (source) release (m) [0053] In this variant: - "source": is for example a string of characters, an array of a type of scalar data, dynamically allocated memory area ... the list is not exhaustive. - "write ()" is an instruction to write to a memory area, for example by copying a source memory area to a destination memory area such as "memcopy" or "memmove". [0054] Another variant would be: sdt = now () total = 0 source = array [] as now () - sdt <duration if total <throughput * (now () - sdt) m = allocation (level) write ( source, m) release (m) total = total + size (source) otherwise sleep () [0055] These variants make it possible to perform several allocation / allocation cycles during the duration of the disturbance and thus to be closer to the behavior from a third party program.
    权利要求:
    Claims (9)
    [0001]
    REVENDICATIONS1. A method for producing controlled disturbances of the activity of an automated processing device during a test scenario of an application implemented by the automated processing device, characterized in that it comprises the following steps: - Acquisition ( 500) of a configuration defining at least one perturbation by: - an identifier of the device to be disturbed, - a start date of the disturbance, - a duration for the disturbance, - a type for the perturbation, - a disturbance level, - Establishment (510) of at least one disturbance generator according to the configuration of the at least one disturbance - Launch (520) of the test scenario.
    [0002]
    2. Process for producing controlled disturbances according to claim 1, characterized in that the type of a disturbance is among the list consisting of at least: - processor load, - read / write disk, - read / write network, - Memory Usage.
    [0003]
    3. Disturbance production method according to one of the preceding claims characterized in that, depending on the type of the disturbance, it comprises a calibration step of the disturbance generator.
    [0004]
    4. Method according to one of the preceding claims, characterized in that a disturbance start date is relative to a start date of the test scenario.
    [0005]
    5. Disturbance production method according to one of the preceding claims characterized in that the generator is a process that knows its start date, has access to the current date and is controlled by a loop structure dependent on the comparison of the difference between the current date and the start date with the duration of the disturbance.
    [0006]
    6. Method of producing disturbance according to claim 5, characterized in that the loop comprises instructions for: - Disrupt the device according to the type of disturbance, - temporarily inhibit the instructions to disturb in order to meet the required level of disturbance.
    [0007]
    7. Disturbance production method according to one of the preceding claims, characterized in that the establishment of the generator on a device comprises at least the following steps: - Copy (510.2) of at least one executable file on the device , - Planning (510.3), on the device, the execution of the at least one copied executable file.
    [0008]
    8. Digital storage device comprising a file corresponding to instruction codes implementing the method according to one of the preceding claims.
    [0009]
    9. Device implementing the method according to one of claims 1 to 7.
    类似技术:
    公开号 | 公开日 | 专利标题
    EP2975526B1|2018-09-19|Method for producing controlled disturbances of the activity of an automated processing device during an application testing scenario
    CN107710215A|2018-02-16|The method and apparatus of mobile computing device safety in test facilities
    US8918371B1|2014-12-23|Systems and methods for event log compensation
    FR3017725A1|2015-08-21|METHOD OF DEPLOYING A SOFTWARE APPLICATION SET |
    US9940115B2|2018-04-10|Mobile application deployment for distributed computing environments
    TW202101199A|2021-01-01|System and method for executing different types of blockchain contracts
    US10063409B2|2018-08-28|Management of computing machines with dynamic update of applicability rules
    US20170262362A1|2017-09-14|Systems and methods for benchmark based cross platform service demand prediction
    WO2011117528A1|2011-09-29|Method, computer program, and device for validating task execution in scaleable computer systems
    US9547652B2|2017-01-17|System and method for validating replication of mutable operations performed on a file system object during a migration
    US10216505B2|2019-02-26|Using machine learning to optimize minimal sets of an application
    EP2996036A1|2016-03-16|Method for monitoring an application architecture comprising a plurality of services
    EP3620928A1|2020-03-11|Device and method for analysing the behaviour of an application component subject to increasing scarcity of resources
    EP2798498B1|2021-02-03|Performance measurement of a computing infrastructure
    FR3069933B1|2019-10-25|METHOD FOR VALIDATION OF MUTUALIZATION OF APPLICATION BRICKS ON A COMPUTER INFRASTRUCTURE
    FR3028973A1|2016-05-27|SYSTEM AND METHOD FOR PERFORMANCE TESTING OF A COMPUTER INFRASTRUCTURE
    US20180331883A1|2018-11-15|Modifying computer configuration to improve performance
    CN110502242A|2019-11-26|Code automatic generation method, device, computer equipment and storage medium
    CN110263548A|2019-09-20|A kind of web application hole detection rule generating method, terminal and storage medium
    FR3012896A1|2015-05-08|METHOD FOR VALIDATION OF AN APPLICATION RESPONSE TIME, METHOD FOR DEPLOYING AN APPLICATION COMPRISING SUCH A VALIDATION METHOD, COMPUTER PROGRAM AND CORRESPONDING DEVICES
    Gonzalez2017|Implementing Modern DevOps: Enabling IT organizations to deliver faster and smarter
    EP3483733B1|2020-05-13|Method and apparatus for searching for a resource optimisation margin of a business chain
    US20210232388A1|2021-07-29|Software pipeline configuration
    EP3195113B1|2019-12-11|Method for verifying traceability of first instructions in a procedural programming language generated from second instructions in a modelling language
    EP3767476A1|2021-01-20|Device and method for analysing the performance of an n-tier application
    同族专利:
    公开号 | 公开日
    FR3023940B1|2017-12-29|
    EP2975526B1|2018-09-19|
    ES2694149T3|2018-12-18|
    EP2975526A1|2016-01-20|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20090156314A1|2007-12-18|2009-06-18|Electronics And Telecommunications Research Institute|System and method for re-generating packet load for load test|
    WO2013098500A1|2011-12-29|2013-07-04|Bull Sas|Performance measurement of a computing infrastructure|
    WO2013145628A1|2012-03-30|2013-10-03|日本電気株式会社|Information processing device and load test execution method|
    US20150058478A1|2012-03-30|2015-02-26|Nec Corporation|Information processing device load test execution method and computer readable medium|
    US20130282334A1|2012-04-18|2013-10-24|w2bi. Inc.|Logistics of stress testing|
    US8726243B2|2005-09-30|2014-05-13|Telecom Italia S.P.A.|Method and system for automatically testing performance of applications run in a distributed processing structure and corresponding computer program product|
    US8010325B2|2008-04-25|2011-08-30|Microsoft Corporation|Failure simulation and availability report on same|FR3085771B1|2018-09-07|2020-12-11|Bull Sas|DEVICE AND METHOD FOR ANALYSIS OF THE BEHAVIOR OF AN APPLICATION BRICK SUBJECT TO A RAREFACTION OF RESOURCES|
    FR3087282B1|2018-10-11|2020-10-09|Bull Sas|PROCESS FOR OPTIMIZING THE MINIMUM SIZING OF AN INFRASTRUCTURE INTENDED TO EXECUTE A SCHEDULE PLAN|
    EP3640800A1|2018-10-17|2020-04-22|Bull Sas|Method for improving the efficiency of use of the resources of an infrastructure designed to execute a scheduling plan|
    FR3087556A1|2018-10-17|2020-04-24|Bull Sas|METHOD FOR IMPROVING THE EFFICIENCY OF USE OF THE RESOURCES OF AN INFRASTRUCTURE FOR IMPLEMENTING A SCHEDULING PLAN|
    FR3091376B1|2018-12-31|2020-12-11|Bull Sas|Process for optimizing a scheduling plan and the sizing of an IT infrastructure|
    法律状态:
    2015-06-25| PLFP| Fee payment|Year of fee payment: 2 |
    2016-01-22| PLSC| Publication of the preliminary search report|Effective date: 20160122 |
    2016-06-22| PLFP| Fee payment|Year of fee payment: 3 |
    2017-06-21| PLFP| Fee payment|Year of fee payment: 4 |
    2018-06-21| PLFP| Fee payment|Year of fee payment: 5 |
    2019-07-25| PLFP| Fee payment|Year of fee payment: 6 |
    2020-07-28| PLFP| Fee payment|Year of fee payment: 7 |
    2021-07-26| PLFP| Fee payment|Year of fee payment: 8 |
    优先权:
    申请号 | 申请日 | 专利标题
    FR1456868A|FR3023940B1|2014-07-17|2014-07-17|METHOD FOR PRODUCING CONTROLLED INTERFERENCE OF AN AUTOMATED PROCESSING DEVICE DURING A TEST SCENARIO OF AN APPLICATION|FR1456868A| FR3023940B1|2014-07-17|2014-07-17|METHOD FOR PRODUCING CONTROLLED INTERFERENCE OF AN AUTOMATED PROCESSING DEVICE DURING A TEST SCENARIO OF AN APPLICATION|
    ES15177094.8T| ES2694149T3|2014-07-17|2015-07-16|Procedure for producing controlled disturbances of the activity of an automated processing device during a test scenario of an application|
    EP15177094.8A| EP2975526B1|2014-07-17|2015-07-16|Method for producing controlled disturbances of the activity of an automated processing device during an application testing scenario|
    [返回顶部]